This application is related to an application filed the same date and titled xe2x80x9cImage Processing Method and Image Processing Device,xe2x80x9d attorney docket no. JAO 36154A.
1. Field of the Invention
The present invention relates to an image processing method and an image processing device which evaluates objectively the copy quality of a copy machine, for example, enables restoration of a deteriorated image, and further may be used as a preparatory process for an optical character recognition (OCR) process.
2. Description of Related Art
The use of copy machines has spread widely in recent times. The capability of recent copy machines to restore images faithfully is improving rapidly. Digital copy machines, in particular, which are becoming more common today, can produce high quality copies with highly faithful copying capabilities.
Therefore, if an original to be copied has a clear and clean image, digital copy machines, with their highly faithful copying capabilities, can produce high quality copies that are virtually the same as the original. However, when the original is one that was copied by a conventional analog copy machine and the image has deteriorated, a digital copy machine, with its highly faithful copying capabilities, will reproduce a deteriorated copy.
Moreover, in recent years, optical character recognition technology is spreading in which a printed text is read by a scanner and characters from the input image data are extracted and changed into computer code. In such optical character recognition technology, a high ratio of recognition is achieved when a text is read by a scanner if the text is an original having a clear and clean character image as described in FIG. 41(a) rather than a copy produced by a copy machine.
However, even if the original has a clear and clean image, the image deteriorates with repeated copying and, if such a deteriorated image is read by a scanner, the copy may result in deteriorated images as described in FIG. 41(b) and FIG. 41(c). FIG. 41(b) represents an image produced by reading a once-copied original by a scanner, while (c) represents an image produced by reading the copy of a once-copied original (twice-copied original) by the scanner.
In order to cope with the deterioration of image quality as a result of copying, various measures have been provided for an optical character recognition mechanism. A dictionary for character recognition, including deteriorated character images, may be prepared and/or a function to repair some degree of unevenness provided.
As described above, a high level of capability to reproduce a faithful copy is required of copy machines, but when a deteriorated image (FIG. 41(b) and FIG. 41(c)) as a result of repeated copying is copied by a digital copy machine with a high level of capability to produce a faithful copy, there have been problems of inability to produce a high quality copy.
Moreover, evaluation of copy machine capability (evaluation of image quality) has been primarily performed by subjective evaluation. In other words, in general, human eyes, after seeing the copied image, determine the capability of the copy machine. Traditionally, there have been technologies associated with copy machines to correct the unevenness of deteriorated characters to some degree when characters which have deteriorated as a result of repeated copying are used as the object of optical character recognition or further copying, but there has been no technology to enable objective, numerical evaluation of image quality, to determine the quality of image, especially the so-called faintness or smudge of the characters, and to repair appropriately according to the deterioration condition of the image.
Moreover, in optical character recognition technology to extract and change into codes the characters from input image data, deteriorated characters are not sufficiently recognized. For example, suppose the character images of FIGS. 41(a), 41(b), and 41(c) were targeted for character recognition. Assuming the character recognition rate of FIG. 41(a) to be 100%, the rate drops rapidly with 90% for 41(b) and 80% for 41(c). Thus, a sufficient recognition rate is not obtained for deteriorated images. Moreover, images pleasing to the eye are not produced.
Furthermore, in assigning binary values to character images, it is necessary to determine the optimum binary threshold values, but the traditional binary threshold determination method does not result in optimum binary threshold values for characters. There have been cases in which faintness and smudges resulted after binary values were assigned to character images by final binary threshold values.
In order to solve these and other problems, the present invention provides an image processing method and an image processing device which, applied to a copy machine, evaluates the quality of the copied image objectively, determines deteriorated sections of the image, enables execution of an improvement process for the deteriorated sections according to the deterioration condition, enables determination of optimum binary threshold values for characters, and may be used effectively as a preparatory process for an optical character recognition device and the like.
An image processing method of the present invention comprises an image quality computation process wherein a characteristic amount is extracted to determine the image quality of image data entered by an image input device, and wherein the characteristic amount is computed as an evaluation value, and the image quality is determined by the evaluation value obtained by the image quality computation process.
Moreover, said computation process may include a first characteristic amount extraction process which has, to begin with, several patterns of pixel characteristics as characteristic points; computes the first characteristic amount, which is the ratio of the frequency of the appearance of said characteristic points in processing lines and the frequency of reversal of black pixels and white pixels; and determines image quality using the first evaluation value which is the first characteristic amount computed above.
Furthermore, said image quality computation process may include a second characteristic amount extraction process which computes the average length of a continuous string of black pixels nearly equivalent to the size of a character; computes the number of continuous strings of black pixels longer than the average continuous string of black pixels nearly equivalent to the size of a character; computes a second characteristic amount which is the ratio of the number of continuous strings of black pixels longer than the average length and one-half of the number of reversals of black pixel and white pixels; and determines image quality using a second evaluation value which is the second characteristic amount computed above.
In addition, said image quality computation process may include the first characteristic amount extraction process and said second characteristic amount extraction process; compute evaluation values based on a first evaluation value obtained by the first characteristic amount extraction process and a second evaluation value obtained by the second characteristic amount extraction process; and the image quality may be determined using an evaluation value based on the first and the second evaluation values.
Moreover, said image quality computation process may include a third characteristic amount extraction process to extract as the third characteristic amount the average length of a continuous string of black pixels nearly equivalent to a character in the processing line and in said first characteristic amount extraction amount; obtain an evaluation value based on a first evaluation value obtained from the first characteristic amount extraction process and a third characteristic amount obtained from the third characteristic amount extraction process; and the image quality may be determined using an evaluation value based on the first evaluation value and the third evaluation value.
Furthermore, said image quality computation process may include the second characteristic amount extraction process and the third characteristic amount extraction process; compute an evaluation value based on a second evaluation value obtained by the second characteristic amount extraction process and a third characteristic amount obtained by the third characteristic amount extraction process; and the image quality may be determined using an evaluation value based on the first and the second evaluation values as well as the third characteristic amount.
In addition, said image quality computation process may include the first characteristic amount extraction process, the second characteristic amount extraction process, and the third characteristic amount extraction process; compute evaluation values based on a first evaluation value obtained by the first characteristic amount extraction process, a second evaluation value obtained by second characteristic amount extraction process, and a third characteristic amount obtained by the third characteristic amount extraction process; and the image quality may be determined using evaluation values based on the first and the second evaluation values as well as the third characteristic amount.
Moreover, the image quality computation process may include a fourth characteristic amount extraction process which performs orthogonal transformation of input image data into frequency regions in order to enable extraction of characteristic amount in the frequency space, computes a fourth characteristic amount by focusing on the high frequency component after orthogonal transformation, and determines the image quality using a fourth evaluation value which is the fourth characteristic amount computed above.
Furthermore, the image quality computation process may include a fifth characteristic amount extraction process which performs orthogonal transformation of input image data into frequency regions in order to enable extraction of characteristic amount in the frequency space, computes a fifth characteristic amount by focusing on the low frequency component after orthogonal transformation, and determines the image quality using a fifth evaluation value which is the fifth characteristic amount computed above.
In addition, if the image quality computation process includes the fourth characteristic amount extraction process and the fifth characteristic amount extraction process, evaluation values may be computed based on a fourth evaluation value obtained by the fourth characteristic amount extraction process and a fifth evaluation value may be obtained by the fifth characteristic amount extraction process, and the image quality may be determined using evaluation values based on the fourth and the fifth evaluation values.
Moreover, the image quality computation process may include the fourth characteristic amount extraction process and the third characteristic amount extraction process, and compute evaluation values based on a fourth evaluation value obtained by the fourth characteristic amount extraction process and a third characteristic amount obtained by the third characteristic amount extraction process, enabling also to determine the image quality using evaluation values based on the fourth evaluation value and the third characteristic amount.
Furthermore, the image quality computation process may include the fifth characteristic amount extraction process and the third characteristic amount extraction process, and compute evaluation values based on a fifth evaluation value obtained by the fifth characteristic amount extraction process and a third characteristic amount obtained by the third characteristic amount extraction process, enabling also to determine the image quality using evaluation values based on the fifth evaluation value and the third characteristic amount.
In addition, the image quality computation process may include the fourth characteristic amount extraction process, the fifth characteristic amount extraction process, and the third characteristic amount extraction process; compute evaluation values based on a fourth evaluation value obtained by the fourth characteristic amount extraction process, a fifth evaluation value obtained by the fifth characteristic amount extraction process, and a third characteristic amount obtained by the third characteristic amount extraction process; and the image quality may be determined using evaluation values based on the fourth and fifth evaluation values as well as the third characteristic amount.
Moreover, the image quality computation process may impose restrictions on the range of extracting said characteristic amount if different regions exist in an original to be processed and compute evaluation values by performing extraction of a characteristic amount for each region.
Furthermore, the image processing method of the present invention may include an image quality computation process to extract a characteristic amount to determine the image quality of image data entered by an image input device and to compute an evaluation value which is the extracted characteristic amount, and an image quality improvement process to determine, from the deterioration characteristic, candidates for the image quality improvement process by extracting sections which have the possibility of deteriorated image quality, and to execute the image quality improvement process on candidates for image quality improvement processing by using evaluation values obtained by the image quality computation process.
The image quality improvement process may include a processing candidate extraction process to extract candidates for image quality improvement processing, and pixel processing to interpolate pixels in executing image quality improvement on processing candidates extracted by the processing candidate extraction process.
The processing candidate extraction process may include a characteristic point extraction process to detect and extract characteristic points generated by deterioration in a section with deteriorated image quality, and a candidate determination process to determine candidates for image quality improvement using the positional relationship of the characteristic points extracted by the characteristic point extraction process.
Moreover, the pixel processing may include a threshold computation process which, using the evaluation value obtained by said image quality computation process, obtains a threshold value from a function with the evaluation value as variable, compares the threshold value with the interval on which interpolation of pixels is performed, and determines whether or not to perform the interpolation process of pixels based on the results of the comparison.
Furthermore, the pixel processing may include a character cutting-out process, also enabling execution of pixel interpolation within the region of characters which are cut out by the character cutting-out process to improve image quality.
In addition, the image processing method of the present invention may include an image quality computation process to extract a characteristic amount to determine the image quality of the image data entered by an image input device and to compute the characteristic amount as an evaluation value, and a binary threshold determination process to determine the binary threshold value for the image to be processed using the evaluation value obtained by the image quality computation process.
The binary threshold value determination process may define a threshold value which determines an evaluation value, based on more than one value among all the evaluation values obtained by said image quality computation process, corresponding to a predetermined value as the target binary threshold value.
Moreover, the image processing method of the present invention may include an image quality computation process to extract a characteristic amount to determine the image quality of the image data entered by an image input device and to compute the characteristic amount as an evaluation value, a binary threshold determination process to determine a binary threshold value for an image to be processed using an evaluation value obtained by the image quality computation process, and an image quality improvement process to determine candidates for the image quality improvement process by extracting sections which have the possibility of image quality deterioration based on the characteristics and to perform the image quality improvement process on candidates for image quality improvement processing using the evaluation value obtained by said image quality computation process.
Furthermore, the image processing device of the present invention may include an image input device to enter images written on originals and the like, and an image quality computation unit to extract and compute a characteristic amount as an evaluation value to determine the image quality of the image data entered by said image input device.
The computation unit may include a first characteristic amount extraction device which has, to begin with, several patterns of pixel characteristics as characteristic points; computes the first characteristic amount, which is the ratio of the frequency of the appearance of said characteristic points in processing lines and the frequency of reversal of black pixels and white pixels; and determines image quality using the first evaluation value which is the first characteristic amount computed above.
Furthermore, the image quality computation unit may include a second characteristic amount extraction device which computes the average length of a continuous string of black pixels nearly equivalent to the size of a character; computes the number of continuous strings of black pixels longer than the average continuous string of black pixels nearly equivalent to the size of a character; computes a second characteristic amount which is the ratio of the number of continuous strings of black pixels longer than the average length and one-half of the number of reversals of black pixel and white pixels; and determines image quality using a second evaluation value which is the second characteristic amount computed above.
In addition, the image quality computation unit may include the first characteristic amount extraction device and the second characteristic amount extraction device; compute evaluation values based on a first evaluation value obtained by the first characteristic amount extraction device and a second evaluation value obtained by the second characteristic amount extraction device; and determine the image quality using an evaluation value based on the first and the second evaluation values.
Moreover, the image quality computation unit may include a third characteristic amount extraction device to extract as the third characteristic amount the average length of a continuous string of black pixels nearly equivalent to a character in the processing line and in the first characteristic amount extraction amount; obtain an evaluation value based on a first evaluation value obtained from the first characteristic amount extraction device and a third characteristic amount obtained from the third characteristic amount extraction device; and the image quality may be determined using an evaluation value based on the first evaluation value and the third evaluation value.
Furthermore, the image quality computation unit may include the second characteristic amount extraction device and the third characteristic amount extraction device; compute an evaluation value based on a second evaluation value obtained by the second characteristic amount extraction device and a third characteristic amount obtained by the third characteristic amount extraction device; and the image quality may be determined using an evaluation value based on the first and the second evaluation values as well as the third characteristic amount.
In addition, the image quality computation unit may include the first characteristic amount extraction device, the second characteristic amount extraction device, and the third characteristic amount extraction device; compute evaluation values based on a first evaluation value obtained by the first characteristic amount extraction device, a second evaluation value obtained by second characteristic amount extraction device, and a third characteristic amount obtained by the third characteristic amount extraction device; and the image quality may be determined using evaluation values based on the first and the second evaluation values as well as the third characteristic amount.
Moreover, said image quality computation unit may include a fourth characteristic amount extraction device which performs orthogonal transformation of input image data into frequency regions in order to enable extraction of characteristic amount in the frequency space, computes a fourth characteristic amount by focusing on the high frequency component after orthogonal transformation, and determines the image quality using a fourth evaluation value which is the fourth characteristic amount computed above.
Furthermore, the image quality computation unit may include a fifth characteristic amount extraction device which performs orthogonal transformation of input image data into frequency regions in order to enable extraction of characteristic amount in the frequency space, computes a fifth characteristic amount by focusing on the low frequency component after orthogonal transformation, and determines the image quality using a fifth evaluation value which is the fifth characteristic amount computed above.
In addition, the image quality computation unit may include the fourth characteristic amount extraction device and said fifth characteristic amount extraction device and evaluation values may be computed based on a fourth evaluation value obtained by the fourth characteristic amount extraction device and a fifth evaluation value may be obtained by the fifth characteristic amount extraction device, and the image quality may be determined using evaluation values based on the fourth and the fifth evaluation values.
Moreover, the image quality computation unit may include the fourth characteristic amount extraction device and the third characteristic amount extraction device, compute evaluation values based on a fourth evaluation value obtained by the fourth characteristic amount extraction device and a third characteristic amount obtained by the third characteristic amount extraction device, enabling also to determine the image quality using evaluation values based on the fourth evaluation value and the third characteristic amount.
Furthermore, the image quality computation unit may include the fifth characteristic amount extraction device and the third characteristic amount extraction device, and compute evaluation values based on a fifth evaluation value obtained by the fifth characteristic amount extraction device and a third characteristic amount obtained by the third characteristic amount extraction device, enabling also to determine the image quality using evaluation values based on the fifth evaluation value and the third characteristic amount.
In addition, the image quality computation unit may include the fourth characteristic amount extraction device, the fifth characteristic amount extraction device, and the third characteristic amount extraction device; compute evaluation values based on a fourth evaluation value obtained by the fourth characteristic amount extraction device, a fifth evaluation value obtained by the fifth characteristic amount extraction device, and a third characteristic amount obtained by the third characteristic amount extraction device; and the image quality may be determined using evaluation values based on the fourth and fifth evaluation values as well as the third characteristic amount.
Moreover, the image quality computation unit may impose restrictions on the range of extracting said characteristic amount if different regions exist in an original to be processed and compute evaluation values by performing extraction of a characteristic amount for each range.
Furthermore, the image processing method of the present invention may include an image input device to enter image written on originals and the like and an image quality computation unit to extract a characteristic amount to determine the image quality of image data entered by an image input device and to compute an evaluation value which is the extracted characteristic amount, and an image quality improvement unit to determine, from the deterioration characteristic, candidates for the image quality improvement processing by extracting sections which have the possibility of deteriorated image quality, and to execute the image quality improvement processing on candidates for image quality improvement processing by using evaluation values obtained by said image quality computation unit.
The image quality improvement unit may include a processing candidate extraction device to extract candidates for image quality improvement processing, and pixel processing means to interpolate pixels in executing image quality improvement on processing candidates extracted by the processing candidate extraction device.
The processing candidate extraction device may include a characteristic point extraction device to detect and extract characteristic points generated by deterioration in a section with deteriorated image quality, and a candidate determination device to determine candidates for image quality improvement using the positional relationship of the characteristic points extracted by the characteristic point extraction device.
Moreover, the pixel processing may include a threshold computation means which, using the evaluation value obtained by said image quality computation unit, obtains a threshold value from a function with the evaluation value as variable, compares said threshold value with the interval on which interpolation of pixels is performed, and determines whether or not to perform the interpolation process of pixels based on the results of the comparison.
Furthermore, pixel processing means may include a character cut-out device, also enabling execution of pixel interpolation within the region of characters which are cut out by the character cut-out device to improve image quality.
In addition, the image processing device of the present invention may include an image quality computation unit to extract a characteristic amount to determine the image quality of the image data entered by an image input device and to compute the characteristic amount as an evaluation value, and a binary threshold determination means to determine the binary threshold value for the image to be processed using the evaluation value obtained by the image quality computation unit.
The binary threshold value determination unit may define a threshold value which makes the evaluation value, based on more than one value among all the evaluation values obtained by said image quality computation unit, correspond to the predetermined value as the target binary threshold value.
Moreover, the image processing device of the present invention may include an image input device to enter images written on originals and the like, an image quality computation unit to extract a characteristic amount to determine the image quality of the image data entered by the image input device and to compute the characteristic amount as the evaluation value, an image quality improvement unit to determine candidates for image quality improvement unit by extracting sections which have the possibility of image quality deterioration based on the characteristics and to perform image quality improvement on candidates for image quality improvement processing using the evaluation value obtained by said image quality computation unit, and a binary threshold value determination unit to determine a binary threshold value for the image to be processed using an evaluation value obtained by the image quality computation unit.
The present invention includes an image quality computation unit to determine the image quality of image data entered by an image input device and to compute the result of the determination as an evaluation value corresponding to the image quality. The image quality computation unit may include a first characteristic amount extraction device which has, to begin with, several patterns of pixel characteristics as characteristic points; computes a first characteristic amount which is the ratio of the frequency of the appearance of said characteristic points in the processing lines and the frequency of the reversal of black pixels and white pixels; and determines the image quality using a first evaluation value which is the first characteristic amount computed above. Moreover, the image quality computation unit may include a second characteristic amount extraction device which computes the average length of a continuous string of black pixels nearly equivalent to the size of a character, computes the number of continuous strings of black pixels longer than the average continuous string of black pixels nearly equivalent to the size of a character, computes a second characteristic amount which is the ratio of the number of continuous strings of black pixels longer than the average length and one-half of the number of reversals of black pixels and white pixels, and determines the image quality using a second evaluation value which is the second characteristic amount computed above.
The image quality computation unit may compute as the first evaluation value said first characteristic amount obtained by the first characteristic amount extraction device, and as the second evaluation value said second characteristic amount obtained by the second characteristic amount extraction device; and obtain an evaluation value, based on the first and the second evaluation values, which may be used for determination of the image quality. Moreover, said image quality computation unit may include a first characteristic amount extraction device and a third characteristic amount extraction device. The third characteristic amount extraction device may obtain as the third characteristic amount the average length of a continuous string of black pixels nearly equivalent to a character in the process line, and obtain an evaluation value, based on the first evaluation value and the third characteristic amount, which may be used to determine the image quality.
Moreover, the image quality computation unit may use the second characteristic amount extraction device and a third characteristic amount extraction device, and compute an evaluation value, based on the second evaluation value and the third characteristic amount, which is used to determine the image quality.
Furthermore, the image quality computation unit may use the first characteristic amount extraction device, the second characteristic amount extraction device, and the third characteristic amount extraction device; and compute an evaluation value, based on the first evaluation value, the second evaluation value, and a third characteristic amount obtained by said third characteristic amount extraction device, which is used to determine the image quality.
Moreover, the image quality computation unit may include a fourth characteristic amount extraction device; perform orthogonal transformation of the input image data into frequency regions in order to enable extraction of characteristic amounts in frequency space; compute a fourth characteristic amount by focusing on the high frequency component after the orthogonal transformation to determine the image quality using a fourth evaluation value, which is the fourth characteristic amount, and a fifth characteristic amount extraction device which performs orthogonal transformation of the input image data into frequency regions; computes a fifth characteristic amount by focusing on the low frequency component after the orthogonal transformation; determine the image quality using a fifth evaluation value which is the fifth characteristic amount computed above; and further make it possible to obtain an evaluation value based on the fourth evaluation value obtained by the fourth characteristic amount extraction device and the fifth evaluation value obtained by the fifth characteristic amount extraction device, and to determine the image quality using the evaluation value based on the fourth and the fifth evaluation values.
More specifically, the first characteristic amount and the fourth characteristic amount indicate the faintness of the characters, while the second characteristic amount and the fifth characteristic amount indicate smudging of the characters. From each of the characteristic amounts representing faintness and smudging, evaluation values associated with faintness and smudging are obtained and the image quality is determined objectively and accurately using these evaluation values individually or jointly.
Moreover, the image quality improvement unit may detect and extract, by the processing candidate extraction device, characteristic points in the image deteriorated sections produced by deterioration, and determines candidates for the image quality improvement process using a positional relationship among the characteristic points extracted by the characteristic point extraction device. Then, by said pixel processing work means, threshold values are obtained using the evaluation values obtained by said image quality computation unit and functions with these evaluation values as variables. Next, the threshold values are compared with intervals on which interpolation of pixels is to be performed and determination is made from the comparison results as to whether or not interpolation of pixels is to be conducted.
By these means, deteriorated sections such as faintness and smudging of characters may be determined accurately. For a section with missing pixels thus determined as a deteriorated section, the interval in which pixels are missing is compared with a predetermined threshold value and interpolation is performed after determining if such an interpolation of pixels is necessary. Thus, a restoration process matching the image quality is realized.
Moreover, a characteristic amount may be extracted and computed as an evaluation value after determining the image quality of the image data read in by an image input device and the threshold value, corresponding to a section in which an evaluation value, based on the two evaluation values associated with said faintness and smudging obtained above and matching a certain predetermined value, may be specified as the binary threshold value to be obtained. Thus, the binary threshold value enabling the best image quality may be determined as the binary threshold value to be obtained, and the binary change best suited for the character may be realized.
Furthermore, by combining the image quality computation process, which extracts a characteristic amount by determining the image quality of the image data read in by said image input device and computes the characteristic amount as the evaluation value; the binary threshold value determination process, which determines the binary threshold value for the image to be processed using the evaluation value obtained by the image quality computation process; and the image improvement process, which determines candidates for the image quality improvement process by extracting a section with possible deterioration of image quality based on its characteristics and performs the image quality improvement process on the candidates for image quality improvement process using the evaluation value obtained by said image quality computation process; objective and accurate determination of the image quality as a result of faintness and smudging may be realized. Moreover, the binary threshold value to be obtained may be determined by an evaluation value based on two evaluation values representing the faintness and the smudging, which enables optimum binary processing. In addition, the interpolation process can be performed if interpolation of pixels is found necessary when images after binary processing produce sections with missing pixels due to faintness.